When insufficient training information is available, supervised Extreme Learning Machine (ELM) is difficult to use. Thus applying semi-supervised learning to ELM, a Semi-Supervised ELM (SSELM) framework was proposed. However, it is difficult to find the optimal solution of SSELM due to its nonconvexity and nonsmoothness. Using combinatorial optimization method, SSELM was solved by reformulating SSELM as a linear mixed integer program. Furthermore, SSELM was used for the direct recognition of medicine and seeds datasets using Near-InfraRed spectroscopy (NIR) technology. Compared with the traditional ELM methods, the experimental results show that SSELM can improve the generation when insufficient training information is available, which indicates the feasibility and effectiveness of the proposed method.